Real time feedback and dynamic adjustment for welding robots

ABSTRACT

Systems and methods for real time feedback and for updating welding instructions for a welding robot in real time is described herein. The data of a workspace that includes a part to be welded can be received via at least one sensor. This data can be transformed into a point cloud data representing a three-dimensional surface of the part. A desired state indicative of a desired position of at least a portion of the welding robot with respect to the part can be identified. An estimated state indicative of an estimated position of at least the portion of the welding robot with respect to the part can be compared to the desired state. The welding instructions can be updated based on the comparison.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/379,741, filed Jul. 19, 2021, which claims priority to U.S.provisional patent Application No. 63/053,324, filed Jul. 17, 2020, bothof which are hereby incorporated herein by reference in their entirety.

TECHNICAL FIELD

This disclosure relates generally to the field of robotic welding. Morespecifically, this disclosure relates to systems and methods forproviding real time feedback and enabling dynamic adjustment for weldingrobots.

BACKGROUND

Welding robots automate the process of welding machine parts.Conventional robotic welding has several challenges. For example,existing methods require a programmer to provide specific instructions(e.g., a welding path) to welding robots to perform robotic welding.Alternatively, handheld devices such as teach pendants can be used toprovide specific instructions to welding robots. Known robotic weldingsystems, however, typically follow welding instructions inflexibly.Therefore, if any changes occur to the welding operation, a portion ofthe welding robot, the seam, or the part itself and/or if theinstructions are not precise, the consequent welding operations canresult in bad welds.

Some existing methodologies use models such as kinematic models orcomputer aided design (CAD) models to determine instructions for thewelding robot. For instance, CAD models can identify an approximatelocation of a seam to be welded relative to the part. Kinematic modelsof the robotics system can be used in part to determine the motion planof the robotic system to move the welding tip in order to depositmaterial in the desired location. However, these models are not alwaysprecise. For example, CAD models may not match the physical dimensionsof the part. More specifically, the actual geometric shape or dimensionsof the parts may vary slightly from the shape or dimensions of the partsin a CAD model. Such discrepancy can result in bad welds.

Therefore, there is an unmet need for new and improved systems andmethods for providing real time feedback and dynamic adjustment forwelding robots to improve the welds in real time and adjust theinstructions in real time.

SUMMARY

In some embodiments, a computer-implemented method for updating weldinginstructions for a welding robot is described herein. Thecomputer-implemented method can include receiving data of a workspacethat includes a part to be welded via at least one sensor. The data canbe transformed into a point cloud data representing a three-dimensionalsurface of the part via a processor. A desired state indicative of adesired position of at least a portion of the welding robot with respectto the part can be identified based on the point cloud data via theprocessor. An estimated state indicative of an estimated position of atleast the portion of the welding robot with respect to the part can becompared to the desired state via the processor. The weldinginstructions can be updated based on the comparison.

In some embodiments, the estimated state can be based at least in parton a model of the unwelded part. The model of the unwelded part caninclude annotations representing the welding instructions for thewelding robot. In some embodiments, the estimated state of the part canbe based at least in part on the data of the workspace.

In some embodiments, the at least one sensor can include a first sensorand a second sensor. The computer-implemented method can furthercomprise synchronizing the first sensor and the second sensor via theprocessor. In some embodiments, the first sensor can be a laser emitterand the second sensor can be a camera.

In some embodiments, a plurality of sensors can include the at least onesensor. The computer-implemented method can further compromise fusingrespective data received from each sensor of the plurality of sensors togenerate fused data, and determine the estimated state based on thefused data.

In some embodiments, comparing the estimated state to the desired statecan comprise implementing a geometric comparator technique. In someembodiments, the computer-implemented method can further compriseupdating the point cloud data based at least in part on the desiredstate. In some embodiments, identifying the desired state can compriseinputting the point cloud data into an artificial neural network togenerate the desired state.

In some embodiments, updating the welding instructions can includeadjusting a motion of the welding robot. In some embodiments, updatingthe welding instruction can include adjusting a motorized fixtureassociated with the part. In some embodiments, updating the weldinginstructions can include dynamically updating welding parameters in realtime.

In some embodiments, the computer-implemented method can furthercomprise detecting a tack weld in a candidate seam on the part to bewelded and updating welding parameters based at least in part on thedetection of the tack weld. In some embodiments, thecomputer-implemented method can further comprise detecting that a gapassociated with a seam on the part varies and updating weldingparameters based at least in part on detection of the variable gap.

In some embodiments, updating welding instructions can comprise updatingwelding parameters. The updated welding parameters can include at leastone of welder voltage, welder current, duration of an electrical pulse,shape of an electrical pulse, and material feed rate. In someembodiments, receiving the data of the workspace can comprise receivingthe data in real time. The computer-implemented method can furthercomprise monitoring the desired state based at least in part on the datareceived in real time.

In some embodiments, identifying the desired state can further compriseimplementing a particle filter on the point cloud data. In someembodiments, identifying the desired state can further compriseimplementing an online classifier on the point cloud data.

In some embodiments, the computer-implemented method can furthercomprise implementing a particle filter and an online classifier on thepoint cloud data. In some embodiments, the at least one sensor can be alaser emitter. The computer-implemented method can further comprise inresponse to identifying at least one occluded laser line from the laseremitter, implementing a particle filter on the point cloud data toidentify the desired state.

In some embodiments, a system comprising a sensor, a controller, and awelding robot is described herein. The sensor can be configured toobtain data of a workspace. The workspace can include a part to bewelded. A controller can be communicably coupled to the sensor. Thecontroller can be configured to: transform the data to a point clouddata representing a three-dimensional surface of the part, identify adesired state indicative of a desired position of at least a portion ofthe welding robot with respect to the part based on the point clouddata, compare an estimated state indicative of an estimated position ofat least the portion of the welding robot with respect to the part tothe desired state, and generate welding instructions based on thecomparison. The welding robot can implement the welding instructions toweld the part.

In some embodiments, an apparatus is described herein. The apparatus cancomprise a welding head including at least one laser emitter configuredto generate laser lines to sweep an arc of 20 degrees to 60 degrees. Theapparatus can also comprise a receiver configured to receive reflectedportion of the laser lines. The apparatus can also comprise a controllerto generate welding instructions based at least in part on the reflectedportion of the laser lines.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic description of a system for precise welding of apart using a welding robot with real time feedback and dynamicadjustment in real time, according to an embodiment.

FIG. 2 illustrates an example welding head, according to someembodiments.

FIG. 3 shows an illustration of a welding head attached to a weldingrobot, according to some embodiments.

FIG. 4 is a flow diagram of a method for precise welding with real timefeedback and dynamic adjustment in real time, according to anembodiment.

DETAILED DESCRIPTION

Non-limiting examples of various aspects and embodiments of theinvention are described herein and illustrated in the accompanyingdrawings.

Systems and methods for providing real time feedback and dynamicadjustment for welding robots are disclosed herein. Technology describedherein can update instructions provided to a welding robot to performrobotic welding in real time based on real time feedback.

Robots that can be configured to perform operations such as welding,brazing, bonding, a combination thereof, and/or the like are referred toherein as “welding robots.” In contrast to manual operations, weldingrobots provide significant benefits by reducing production time, cuttingcost of labor, improving safety, and conserving materials. “Roboticwelding” refers to operations that can be performed by a welding robot,such as welding, brazing, bonding, and/or the like. “Welding robots” asused and described herein can be autonomous or semi-autonomous robots.

Conventional methods and systems for controlling welding robots haveseveral drawbacks. Typically, a model (e.g., a kinematic model for awelding robot and/or a computer aided design (CAD) model for a part tobe welded) can be used to determine an “estimated state” (described indetail below) or sequence of estimated states of the system. Forinstance, the estimated state could indicate a position of the weldingrobot (e.g., position of a weld tip of the welding robot) with respectto the part to be welded. Conventional technologies often use theestimated state as a starting point for controlling the welding robot.More specifically, robotic weld programs (e.g., software programs,algorithms, etc.) to control a welding robot are generated based on theestimated state of the system. These robotic weld programs include aweld path (e.g., trajectory for the weld tip) for the welding robotbased on the initial state such that the welding robot can perform awelding operation on the part.

However, there are several challenges associated with relying solely onthe model(s). For example, kinematic models may not account for thingslike backlash, imprecise appendage lengths, or bending of componentsincluded in the welding robot. Similarly, CAD models may not includeprecise dimensions of the part to be welded, for example, when an actualpart deviates from a model of that part. Such discrepancies could leadto bad welds. More precisely, the determined estimated state may resultin the welding robot being positioned in an inaccurate/imprecise mannerrelative to the part. That is, the calculated weld path may beinaccurate owing to these discrepancies. Therefore, traditional roboticweld programs can cause poor welds, misplaced welds, collisions amongstcomponents within the system, or other detrimental phenomenon.

Furthermore, most robotic weld programs are often repeatedly implementedon different units or examples of the same part. Put differently, thesame CAD model may be used to determine an estimated state for differentunits of the same part. Therefore, inconsistencies between differentunits can lead to bad welds. Similarly, unexpected part placementrelative to the welding robot, misaligned parts, poor tolerances, orother similar issues can lead to bad welds.

Additionally, in some situations, a part may shift, move, or deformduring a welding process. More specifically, welding processes oftenproduce high levels of internal stresses to parts which could induce theparts to shift, deform, or otherwise move during a welding process. Suchshifts can cause robotic weld programs that are based on estimated stateto be imprecise and/or inaccurate thereby causing bad welds. Inaddition, during processes such as additive manufacturing process orcladding, multiple layers of material may need to be deposited on apart. Accordingly, multiple passes of weld may be needed in order todeposit multiple layers. The estimated state in such a case maytherefore change after a single weld pass.

Thus, real time feedback during welding operations can reduce bad welds.Moreover, using real time feedback to dynamically adjust weldinginstructions for the welding robots can result in precise welding ofparts. However, the accuracy of systems implementing feedback can varydepending on the amount of data that the systems incorporate intofeedback.

Existing feedback system are often primitive providing merely fineadjustments such as fine adjustments to the welding head or adjustmentsto the welding robot's motion and trajectory so that the welding tip ison the seam to be welded. This does not always ensure that the weld isprecise. For instance, the welding instructions are programed inexisting feedback systems based typically on two-dimensional drawing ormodel of the expected seam. Such two-dimensional models may be providedby a user. The welding instructions may include a weld path (e.g.,motion and/or trajectory) that the welding robot is to traverse based onthe geometry of the seam or gap. Feedback may be provided using alaser-scanner sensor that looks ahead of the welding tip. This can allowexisting feedback systems to determine features the welding tip is aboutto encounter. The laser-scanner looks ahead of the welding tip andprovides feedback that generally allows the weld head to follow a seam,as long as the seam geometry remains consistent and visible to thelaser-scanner.

However, there are several drawbacks associated with such existingfeedback systems. If the modeled geometry of the seam or the gap is notprecise and does not match the actual physical geometry of the seam orthe gap, then such feedback systems may not recognize or localize thegaps or seams. Similarly, some portions of the seam or gap may beoccluded or may not be completely visible to the laser-scanner duringthe welding operations. Therefore, existing feedback systems may notrecognize such occluded portions during welding operations. Furthermore,since existing feedback systems make minor adjustments to the motion ofthe welding robot such systems may not have the ability to adjust motionof motorized fixtures (e.g., fixtures that can be actuated to move apart relative to the welding robot).

Furthermore, in some instances, since the laser-scanner looks ahead todetermine features that the welding tip might encounter, the weldingrobot may be constrained to always point or aim the laser-scanner aheadof the welding operation. These constraints can slow down the system,force a collision when welding a seam, and/or can render a seam to bewelded unweldable.

Therefore, there is a need for sophisticated real time feedback todynamically adjust welding operations in real time.

The technology described herein includes sensors to scan a part. In someembodiments, the technology described herein can determine an estimatedstate of the system, part, welding robot, fixtures, and/or the seam. Theestimated state can be estimated position, estimated velocity, and/orestimated acceleration of the system, part, welding robot, fixtures,and/or seam in a universal or relative reference frame. For example, theestimated state can be an estimated position of the welding robotrelative to the part. The estimated state can be determined based onsensor input and/or models such as CAD models of the part and/orkinematic models of the welding robot.

As described above, the sensors can scan the part, fixtures, portions ofthe welding robot, workspace, etc. The technology described herein canautomatically recognize a seam to be welded on the part based on thesensor data. In some embodiments, the technology described herein canlocalize the seam relative to the part. The technology described hereincan then determine a desired state, or sequence thereof, of the system,part, welding robot, fixtures, and/or the seam based on the recognition(and localization) of the seam. The desired state can be desiredposition, desired velocity, and/or desired acceleration of the system,part, welding robot, fixtures, and/or seam relative to each other. Insome embodiments, the technology described herein can compare thedesired state to the estimated state and can automatically update arobotic weld program based on the comparison. For instance, the roboticweld program can include a motion/trajectory (e.g., weld path) of thewelding robot, welding head, other devices such as motorized fixtures,etc. in order to weld seams on the part precisely as desired withoutcollision prior to the welding operation. An update to the robotic weldprogram can include dynamic real time adjustment to the motion of thewelding robot, welding head, other devices such as motorized fixtures,etc. in order to weld seams on the part precisely as desired withoutcollision during the welding operation.

As discussed above, the technology described herein can adjustmotion/trajectory (e.g., weld path) for a welding robot based on dataobtained from sensors. The adjustment can be made dynamically based onthe comparison of the expected state with the desired state.Accordingly, in contrast to existing feedback systems, the technologydescribed herein does not require a user to provide the shape of theseams or gaps. Additionally, even if a portion of the seam or the gap isoccluded, the technology described herein can still dynamically adjustwelding instructions by automatically recognizing seams and/or gapsbased on the sensor data.

In contrast to existing feedback systems, the technology describedherein can also operate and/or adjust the motion/trajectory of otherdevices such as motorized fixtures. For instance, the motorized fixturescan be actuated to move a part to be welded. Since the technologydescried herein can also move the motorized fixtures, this can lead to aminimization of energy consumed by the system by limiting total motionwithin the system. In addition, this can improve or optimize the weldingprocess by optimizing the position and orientation of the seam or gaprelative to the welding tip and gravity. Additionally, this can alsocurtail collisions or possible collisions.

In contrast to existing feedback systems, the technology describedherein can also adjust welding parameters in real time. This in turn canimprove the quality of the weld. Some non-limiting examples of weldingparameters include welder voltage, welder power or current, materialfeed rate (if applicable), and other parameters like duration of pulseor shape of electrical pulse, and/or the like. Some non-limiting reasonsfor adjusting welding parameters alone or in addition to themotion/trajectory of the welding robot and motorized fixture motioninclude: 1) better welding over or through welds. More specifically,tack welds along a seam usually have material properties that may bedifferent from the material properties of the part(s). Updated weldingparameters can account for the shape and geometry of the tack welds; 2)better filling of variable gap sizes by adjusting the feed rate ofmaterial and other welding parameters necessary to deposit at thatlevel; 3) insuring good penetration of the weld over different seamgeometries; 4) being able to produce the desired weld when for examplecollision issues interfere with the motion of the welding robot and/orother components of the system; and 5) mitigation, elimination, oralteration of porosity, inclusions, lack of fusion, undercut, overlap,insufficient throat, excessive convexity (i.e. excessive reinforcement,underfill, uneven weld legs, weld splatter, root concavity (i.e.suck-back), hot cracking, or cold cracking.

Furthermore, since the technology described herein includes acombination of model-based (e.g., determining the estimated state basedon a model) and automatic recognition-based (determining a desired statebased on the recognition and localization of gaps and seams from sensordata) processing, the technology can operate without a prioriinformation about the shape of the gap or seam.

FIG. 1 is a schematic description of a system 100 for precise welding ofa part 114 using a welding robot 110 with real time feedback and dynamicadjustment in real time, according to an embodiment. A workspace 101 caninclude a part(s) 114 to be welded. The workspace 101 can be anysuitable welding area designed with appropriate safety measures forwelding. For example, workspace 101 can be a welding area located in aworkshop, job shop, manufacturing plant, fabrication shop, and/or thelike. The part(s) 114 can be held, positioned, and/or manipulated in theworkspace 101 using fixtures and/or clamps (collectively referred to as“fixtures” 116 herein). The fixtures 116 can be configured to securelyhold the part 114. The fixtures 116 can induce forces necessary to limitthe movement (e.g., translation and/or rotation) of parts 114 relativeto other components of the system 100. For instance, fixtures 116 can bemechanical, electromechanical (e.g., another robotic system), magnetic,or any other means that can fix and/or restrict the movement of theparts 114 relative to other components of the system 100. In someembodiments, the fixtures can be adjustable. In some embodiments, thefixtures 116 can be motorized. Put differently, the fixtures 116 can beactuated to move part(s) 114 during welding process. Therefore, thefixtures 116 can be actuated in such a manner that the position and/ororientation of part(s) 114 relative to other components of the system100 can be changed. In some embodiments, the fixtures 116 may be locatedfully within the workspace 101. Alternatively, the fixtures 116 may belocated partially within the workspace 101. In another alternativeembodiment, the workspace 101 may not include the fixtures 116.

The welding robot 110 can perform welding operations on the part 114. Insome embodiments, the welding robot 110 can be a six-axis robot with awelding arm. The welding robot 110 can be any suitable robotic weldingequipment such as Yaskawa® robotic arms, ABB® IRB robots, Kuka® robots,and/or the like. The welding robot 110 can be configured to perform arcwelding, resistance welding, spot welding, TIG welding, MAG welding,laser welding, plasma welding, a combination thereof, and/or the like.The welding robot 110 can be responsible for moving, rotating,translating, feeding, and/or positioning welding head, sensor(s) 102,part(s) 114, and/or a combination thereof.

In some embodiments, a welding head can be mounted on, coupled to, orotherwise attached to the welding robot 110. FIG. 2 illustrates anexample welding head 220, according to some embodiments. In someembodiments, the welding head 220 can be a mechanical structure and/oran enclosure that can hold, confine, align, point, or direct at leastsome of the electrical, optical, or sensor(s) of the welding robot 110.In some embodiments, the welding head 220 can be mounted on, coupled to,or otherwise attached to the welding robot 110 in such a manner that theat least some of the sensor(s) can observe the area or region around thelocation of the welds during the welding process. In some embodiments,the welding head 220 can include a processor to process sensorinformation obtained from the sensor(s) included within the welding head220.

In some embodiments, the welding head 220 can include laser emitters 202c. In some embodiments, the laser emitter 202 c can include polarizingfilters. Laser emitters can be laser line(s), laser pattern emitter(s)and/or any other suitable sensing device that is included in, mountedon, or attached to the welding head 220. At least some part of the laserline(s) or pattern(s) can be emitted out of the welding head 220. Insome embodiments, laser emitters 202 c can be configured to generatelaser lines or patterns that sweep and arc of 20 degrees to 60 degrees.Similarly stated, in some embodiments, laser emitters 202 c can projectline(s) and/or pattern(s) in front of, to the sides of, and/or at leastpartially behind the welding head 220. In some embodiments, the laseremitters 202 c can be motorized to generate laser lines or patterns thatsweep across an area of the part 112. This could enable the laseremitters 202 c to scan a large area of the part. For instance, the laseremitter 202 c itself could be configured to rotate to sweep the area.Additionally or alternatively, a mirror positioned in front of thelasers can be motorized to perform the sweeping motion. For example, themirrors can have one or two degrees of freedom.

In some embodiments, the welding head 220 can optionally include lasercontroller(s) that can be operable to control power delivery to thelaser emitters 202 c. The laser controller(s) can be configured to turnthe lasers on and off. The laser controller(s) can be configured totransmit feedback associated with laser emitters to one or moreelectronic devices coupled to laser emitter(s) 202 c. In someembodiments, a receiver can be configured to receive reflected portionsof the laser lines or patterns emitted from the laser emitters 202 c. Insome embodiments, the laser controller(s) can be configured toadditionally function as the receiver.

One or more optical sensor(s) 202 d can be included in the welding head220. Some non-limiting examples of optical sensor(s) 202 d includecameras, lenses, lens filters, any suitable optical device, and/or anysuitable opto-electronic device. In some embodiments, filters can beused for filtering out specific wavelengths of light observable to theoptical sensor(s) 202 d. In some embodiments, filters can protect theoptical sensor(s) 202 d from different light emission during the weldingprocess. In some embodiments, filters can enhance the laser emissionfrom the laser emitter(s) 202 c, for example light polarizing filterscan be used. In some embodiments, the welding head 220 can include atleast two optical sensor(s) 202 d mounted on, attached to, or otherwisecoupled to the welding head 220. The optical sensor(s) 202 d can observethe welding head 220, the welding process, and the laser light emittedfrom laser emitter(s) 202 c.

In some embodiments, the welding head 220 can include thermal camerasand/or thermal sensors that sense temperature. In some embodiments,ambient lights and associated controllers can be used to connect tolight sources that may be disposed and/or otherwise positioned withinthe welding head 220 to illuminate the workspace 101.

The welding head 220 can include air blades 224 and can be used toprotect portions of the welding head 220 from particulates and/oremissions from the welding process. For example, an exit air port can bepositioned near the optical sensor(s) 202 d and/or the laser emitter(s)202 c so as to protect the optical components. Additionally oralternatively, one or more magnets can be positioned within the weldinghead 220 to pull particulates and/or emissions from the welding processaway from the optical components of the optical sensor(s) 202 d and/orthe laser emitter(s) 202 c.

In some embodiments, the welding head 220 can include one or morefiducials, such as reflective markers, LED light emitters, stickers withidentifiable patterns, etc. Such fiducials can be used to track theposition and/or the orientation of the welding head 220 by one or moreglobal sensor(s) (e.g., sensor 102 a and 102 b in FIG. 1 ). By trackingthe position and/or the orientation of the welding head 220, a desiredposition and/or orientation of the welding head 220 can be determined.

In some embodiments, the welding head 220 can include force feedbacksensors to detect force and/or torque applied to the welding head 220.In some embodiments, ultrasonic sensors can be coupled to, mounted on,or otherwise attached to the welding head 220. Ultrasonic sensors canhelp avoid collision. Additionally, ultrasonic sensors can sensesurfaces of objects around the welding head 220 and can therefore beused to determine a desired state of the welding head 220. In someembodiments, inertial measurement unit(s) (e.g., accelerometers and/orgyroscope(s)) can provide an estimate of acceleration and/or orientationof at least a portion of the welding robot 110. The welding head 220 caninclude a welding torch 222 to perform the welding operation.

FIG. 3 shows an illustration of a welding head 320 (e.g., structurallyand/or functionally similar to welding head 220 in FIG. 2 ) attached toa welding robot 310 (e.g., structurally and/or functionally similar towelding robot 110 in FIG. 1 ) by means of clamping to a welding torch322 at the end of the welding robot 310.

Referring back to FIG. 1 , sensors 102 a, 102 b, 102 c, 102 d, . . .(collectively referred to as sensor(s) 102) can capture data associatedwith the workspace 101. In some embodiments, one or more sensor(s) 102can be a mounted to the robot 110 or otherwise can be integral to theworkspace 101. For example, the one or more sensor(s) (e.g., sensors 102c and 102 d in FIG. 1 ) can be attached to or otherwise can be a part ofthe of the welding robot 110. More specifically, the sensor(s) (e.g.,sensors 102 c and 102 d in FIG. 1 ) can be a part of, attached to, ormounted on a welding head (as described in FIG. 2 ) of the welding robot110. In some embodiments, sensors 102 c and 102 d can be laser emitters,optical sensors (e.g., cameras), or a combination thereof.

In some embodiments, one or more global sensor(s) (e.g., sensor 102 aand 102 b in FIG. 1 ) can be mounted on another robot (not shown in FIG.1 ) positioned within the workspace 101. For example, a robot may beoperable to move (e.g., rotational and/or translational motion) suchthat the sensor(s) 102 a and 102 b can capture image data of theworkspace 101 from various angles. In some embodiments, one or moresensor(s) 102 a and 102 b can be portable device such as a handheldcomputer tablet, a smartphone with camera, or a digital camera.Additionally or alternatively, one or more sensor(s) 102 a and 102 b canbe laser emitters.

In some embodiments, the sensor(s) 102 described herein can beconfigured to observe and collect data of all regions including theregion around the welding tip. For example, multiple laser lines (e.g.,from multiple laser emitters) and/or optical sensors can effectivelysurround the welding tip, such as forming a closed polygon.

Sensor(s) 102 can determine information associated with the desiredstate of the part 114, fixtures 116, welding robot 110, and/or the seam.As discussed above, desired state can include the desired positionand/or desired orientation of the part 114, fixtures 116, welding robot110, and/or the seam in the physical space relative to each other.Additionally or alternatively, desired state can include desiredvelocity, acceleration, forces, and/or torques associated with the part114, fixtures 116, welding robot 110, and/or the seam. Sensor data caninclude multiple sets of state such that each state in the set of statescan correspond to a state at a different point in time.

The sensor(s) 102 can be operable to capture 2D and/or 3D images/data ofthe part 114, fixture 116, portion of the robot 110, and/or workspace101. In some instances, the sensors 102 can be operable to image and/ormonitor a weld laid by the robot 110, before, during, and/or after welddeposition. In some embodiments, the sensor(s) 102 can generate a pointcloud representation of the workspace 101.

FIG. 1 illustrates two sensors 102 c and 102 d within the robot 110 andtwo global sensors 102 a and 102 b solely for illustrative purposes. Itshould be readily understood that any suitable number of sensors can beincluded in the robot 110. Similarly, any suitable number of sensors canbe positioned in the workspace 101 as global sensors.

The sensor(s) 102 can be communicatively coupled to the controller 108.The sensor(s) 102 and/or the controller 108 can be operable to processdata from the sensor(s) 102 to assemble two-dimensional data andthree-dimensional data from multiple sensors 102 to generate point clouddata. The controller 108 can include one or more processors (e.g., CPU).The processor(s) can be any suitable processing device configured to runand/or execute a set of instructions or code, and can include one ormore data processors, image processors, graphics processing units,digital signal processors, and/or central processing units. Theprocessor(s) can be, for example, a general purpose processor, a FieldProgrammable Gate Array (FPGA), an Application Specific IntegratedCircuit (ASIC), and/or the like. The processor(s) can be configured torun and/or execute application processes and/or other modules, processesand/or functions associated with the system 100.

In some embodiments, the controller 108 can synchronize the sensor(s)102. For instance, the controller 108 can synchronize optical sensor(s)and laser emitter(s) such that when the sensor(s) capture images of thepart(s) 112 and/or workspace 101, each captured image includes only asingle laser pattern such as a line. Put differently, multiple laserlines originating from different laser emitters can be challenging foroptical sensors to distinguish. Accordingly, the controller 108 can beconfigured to strobe or alternate laser lines such that only a singleline is present at any given time. Similarly, the controller 108 cansynchronize optical sensor(s), welding head, and laser emitter(s) suchthat the laser emitter(s) are turned on and an image of the laserresponse captured when the welding head is not operating in pulse mode(e.g., mode in which an electric arc or electric current that inducesthe substrate, filler material, or both to melt occurs in pulses throughtime). Such synchronization can produce images with less light pollutioninduced by the welding process, for example, by capturing images inbetween periods in which the welder is active (e.g., between arcpulses).

In some embodiments, the controller 108 can process the data obtainedfrom the sensor(s) 102. For example, the controller can identify andgenerate laser lines and/or patterns emitted by laser emitter(s) such asdisclosed in U.S. Pat. No. 10,551,179, the entire disclosure of which ishereby incorporated by reference. The laser lines and/or patterns ascaptured by optical sensors can vary in the presence other components ofthe system 100, shadows, different light sources, light emitted fromwelding processes, etc.

In some embodiments, the controller can generate three-dimensional pointcloud data. Point cloud data can represent three-dimensional surface ofparts 112, fixtures 116, and other components of system 100 that arevisible to the welding head of the welding robot 110. Point cloud datacan be generated, for example, using laser and/or light triangulationtechniques, as Structure-From-Motion, and/or Stereo Based approaches.The three-dimensional data may be collected using laser findingtechniques.

In some embodiments, one or more images (e.g., image data captured bythe sensor(s) 102 at a particular orientation relative to part 114) maybe overlapped together to reconstruct and generate three-dimensionalimage data of the workspace 101. The three-dimensional image data can becollated in a manner such that the point cloud generated from the datacan have six degrees of freedom. For instance, each point on the pointcloud may represent an infinitesimally small position inthree-dimensional space. The sensor(s) 102 can capture multiple imagesof the point from various angles. These multiple images can be collatedto determine an average image pixel for that point. The averaged imagepixel can be attached to the point. For example, if the sensor(s) 102are color cameras having red, green, and blue channels, then the sixdegrees of freedom can be {x-position, y-position, z-position,red-intensity, green-intensity, and blue-intensity}. If the sensor(s)102 are black and white cameras with black and white channels, then fourdegrees of freedom may be generated.

The controller 108 can be configured to perform seam recognition basedon the point cloud data. In some embodiments, the controller 108 canlocalize, classify, and/or measure seams, separating volumes, and/orwelds. In some embodiments, the controller 108 can implement a neuralnetwork to recognize and/or classify a seam to be welded. For instance,the neural network can be configured to implement pixel-wise and/orpoint-wise classification on the point cloud data in order to recognizeand classify a seam to be welded. In this manner, the controller 108 cangenerate recognized seam data that encodes the position of seams,separating volumes, and/or welds. The controller 108 can determine adesired state of the part 112, fixture 116, welding robot 110, and/orseam based on the seam recognition and classification.

In some embodiments, the controller can determine an estimated state ofthe part 112, fixture 116, welding robot 110, and/or seam based on oneor more a priori models (e.g., kinematic models of welding robot 110,CAD models of part 112, etc.). For example, CAD models of the part 112can include annotations representing one or more seams on the part to bewelded. Based on the geometry and/or shape of the part 112 and seams,the controller 108 can determine an estimated state. The controller 108can then compare the estimated state to the desired state. For instance,the controller 108 can implement geometric comparator techniques tocompare the estimated state to the desired state. The controller 108 canthen correct for inaccuracies in data from sensor(s) 102 or stateestimations (e.g., desired state and estimated state estimations), tocorrect for bending, movement, or deformation of part(s) 112 relative tothe system 100 that may be induced by the welding process. In thismanner, the controller 108 can provide dynamic adjustments to weldinginstructions.

Geometric comparator techniques takes into account geometric data (e.g.,expected, modeled, and/or predicted data) of the from part(s) 112,seam(s), etc., measured state information of the part(s) 112, robot 110,fixture(s) 116, etc., and also three-dimensional point cloud data anduses this data. Geometric comparator techniques can use this data togenerate a more accurate estimate of state(s) (e.g., desired state) ofthe part(s) 112, seam(s) 116, robot 110, weld(s), etc. Some non-limitingexamples of geometric comparator techniques generally can includedetermining an estimated state from the models, determining a desiredstate from the three-dimensional point cloud, and measuring andquantizing the difference between the desired state and the estimatedstate.

In some embodiments, an accurate estimate of the desired state can bedetermined by implementing, Kalman filters, particle filters, acombination thereof, and/or the like. For instance, particle filters atsome given time-step, can include several particles representative ofpossible or feasible states. The possible or feasible states can beassociated with simulated point cloud data, stimulated image data, orcombination thereof for each of the particles. The estimated state canthen be determined by comparing the stimulated point cloud data and/orthe stimulated image data (collectively referred as “simulated data”)with the actual point cloud data and/or actual image data (collectivelyreferred as “actual data”). This helps determine the most probableparticle (state) that the system is currently in.

In some embodiments, the controller 108 can implement a neural networksuch as Convolutional Neural Network (CNN), Artificial neural network(ANN) to estimate a desired state. In some embodiments, the neuralnetwork can take image data, point cloud data, or a combination thereof,and output an encoding of the input. For instance, the encoding can beinto a smaller dimensional latent space. In some embodiments, the outputfrom a particle of the particle filter (e.g., most probable particlethat the system is currently in) can be provided as an input to theneural network. For example, a stimulated point cloud data and/or imagedata may be provided as an input to the neural network. The output ofthis input (i.e., the encoding of this input) can be compared to theoutput from the neural network of actual point cloud data and/or actualimage data. For instance, the output of the stimulated data can becompared with the output of the actual data by performing a dot productof the two. Additionally or alternatively, the neural network can inputoutput from one of the particle of the particle filter as well as theactual data at the same time. The neural network can then generate ascore that represents the similarity between the two data. Additionallyor alternatively, any suitable method may be used to compare thesimulated data to the actual data.

In some embodiments, different CNNs can be used to process image datafrom optical cameras, thermal cameras, and/or other sensors.Alternatively, the controller can be configured to fuse image data fromvarious sources. For example, the controller 108 can fuse data fromvarious sensor(s) 102 such as optical sensor(s), laser emitter(s), andother sensor(s). By fusing the data from various sources, the controller108 can provide accurate feedback to the welding robot 110 in order toperform precise welding. For instance, by receiving data of the weldingoperation from two or more sensors can provide a wide-angle view of thepart and the weld tip during the welding process. In contrast, when asingle sensor such as a laser emitter is used to provide data, it ispossible that some portion of the seam that is being welded may beoccluded from the laser emitter. Such occlusion can lead to largedeviations from the desired state. With complete view of the part duringthe welding process, the controller can accurately identify the desiredstate and make updates to it in real time. In some embodiments, in orderto fuse data from various sources, in the CNN where branches of networksconverge are combined. The CNN(s) can be trained, for example, usingcost or error functions, parameter optimization or parameter adjustment,gradient descent methods, and/or any other suitable technique.

In some embodiments, the controller 108 can implement a Neural Networkin order to analyze the weld state (e.g., state of the part(s) 112,fixture(s) 116, welding robot 110, and/or seam during the weldingprocess). For instance, as discussed above a neural network can beimplemented to compare simulated data with the actual data. Additionallyor alternatively, a neural network can be implemented to classify seamsor gaps to be welded. In some embodiments, a neural network can beimplemented to classify tack welds or other welds that exist on theseam. The neural network can be structurally and/or functionally similarto a Artificial Neural Network (ANN), Recurrent Neural Network (RNN)and/or Convolutional Neural Network (CNN) or any other suitable neuralnetwork. For example, ANN elements that perform differentclassification, regression, or generative tasks, may feed into otherANNs or other intermediate processing methods or filters. The ANN can beconfigured to derive, generate, and/or originate weld state informationbased on data originally produced by the welding head, sensor(s) 102, orcombination thereof. The ANN can be trained, for example, using cost orerror functions, parameter optimization or parameter adjustment,gradient descent methods, and/or any other suitable techniques.

In some embodiments, the welding robot 110, part(s) 112, fixtures 116,and/or other components of the system 100 can be controlled by a roboticwelding program executed on the controller 108. The robotic weld programcan be configured to transmit welding instructions to the welding robot110, part(s) 112, fixtures 116, and/or other components of the system100 so as to weld seams or separating volumes. In some embodiments, therobotic weld program can be generated prior to the commencement of thewelding process or execution of the robotic weld program itself. Therobotic weld program can be dynamically altered, interrupted, deleted,modified, or added to by the controller 108 based on comparison of thedesired state and estimated state. For instance, the robotic weldprogram can be updated, created, altered, deleted, amended, or otherwisechanged specific to the motion/trajectory of the welding robot 110and/or other motorized components (e.g., motorized fixture(s) 116)within the system 100. Additionally or alternatively, the robotic weldprogram can be updated, created, altered, deleted, amended, or otherwisechanged specific to the welding process (e.g., angle of the weld tiprelative to the part, velocity of weld tip, acceleration of weld tip,etc.). As discussed above, the controller 108 can update, create, alter,delete, amend, or otherwise change the robotic weld program based on thecomparison of the estimated state to the desired state. For instance,the weld tip may be moved from an estimated position to a more accuratedesired position.

Some non-limiting techniques implemented by the controller 108 to updatethe robotic weld program include a) implementing a linear time invariantcontroller employing control theory methods, tuning, and modeling, thatuses error based on modeled and observed state information for the robot110, fixture(s) 116, part(s) 114, seam, weld, etc.; b) implementing anon-linear classical controller that can be build, for example, oneither Gain Scheduling, Feedback Linearization, or Lyapunov, and/or anyother suitable methods that uses error based on modeled and observedstate information for the robot 110, fixture(s) 116, part(s) 114, seam,weld, etc.; c) implementing optimal motion controller(s), which caninclude any applicable optimal controllers, that uses error based onmodeled and observed state information for the robot 110, fixture(s)116, part(s) 114, seam, weld, etc.; d) implementing artificial neuralnetwork controllers which can be built and trained neural networks thatare constituted based on mathematical elements most typically associatedwith ANNs and can use error based on modeled and observed stateinformation for the robot 110, fixture(s) 116, part(s) 114, seam, weld,etc.

In some embodiments, the controller 108 can be configured to recognizeand measure root gap, hole, void, and weld and update robotic weldprogram based on recognition in order to produce more precise welds. Thetwo-dimensional images and the three-dimensional point cloud data fromthe sensor(s) 102 can be used to recognize the root gap, hole, void, orweld and further can be used to adapt and/or update the robotic weldprogram to account for the root gap, hole, void, or weld. In someembodiments, in a first mode, the controller 108 can first recognize theroot gap, hole, void, or weld and can then subsequently update therobotic weld program. Alternatively, in a second mode, the controller108 can recognize the root gap, hole, void, or weld and can update therobotic weld program simultaneously.

In the first mode, the controller 108 can break down the task ofrecognition, classification, and measurement of root gaps, holes, voids,welds, or any obstruction along the seams separately. This can providean indication on how to adapt and/or update the robotic weld program. Inthis mode, a user can see and understand the decisions that thecontroller 108 makes.

Recognition and measurements of root gaps, holes, voids along the seamcan be accomplished in part or in whole via trained neural networks. Theinput data to these neural networks can be point cloud data and/ortwo-dimensional image data. The data flows into a one or more networkswhich then classifies regions or boundaries of holes, voids, or rootgap. In some embodiments, the neural network can determine the size ofthe holes, voids, or root gap. The input to this neural network mayinclude a model of the part(s) 112 or regions on the part(s) 112. Forexample, the input to the neural network can be a CAD model of a part(s)112 that has had its surface sampled and turned into point cloud. Insome embodiments, these neural networks can be any suitable neuralnetwork such as Dynamic Graph CNN, point-net, etc. The input data can bepresented to the neural network in any suitable form (e.g., as unorderedlist of vectors). The input data can be pre-processed (e.g., usingnormalization).

In some embodiments, the input data can be provided to the neuralnetwork at once. Alternatively, the input data can be provided to theneural network incrementally. The neural networks can be trained in anysuitable manner. In some variations, the neural networks can includelong short-term memory (LSTM) modules, or other similar modules meant todeal with time-varying or sequential data. The output could be in anysuitable form. For example, the output could include classification ofgap associated for each original three-dimensional points and/or also asubset of the three-dimensional points and/or can be associated withpixels of the two-dimensional images. The neural networks can be trainedto output other information simultaneously, for example surfaceroughness measurements, and are not necessarily limited to just handlethe recognition and measurement tasks alone.

Adaptation and/or updating the robotic weld program can also occur inpart via trained neural network(s). This neural network, or set ofneural networks, can take in the resultant classification outputdescribed above, information about what the desired weld is, and whereit should be. This neural network(s) can then output at leastinformation about orientation and/or location of the weld tip relativeto the seam. The network may also be trained such that weld parameters,for example voltage, current, arc-length, wire/wire material, substratematerial and thickness, and orientation of the seam relative to gravitycan be either inputs, outputs, or some combination of both. For example,a user may want to constrain the voltage, current, and arc-length,(inputs the network) but not the orientation of the seam relative togravity (output). Alterations or adaption of weld parameters likecurrent, voltage, or arc-length can be useful for changing the shape ofthe welds or handling transition between substrates like welds that havebeen previously deposited. The neural network may be trained to take ina range, envelope, or bounds of parameters. For example, there could betwo inputs for voltage that bound the allowable range. The network maybe trained to output over a range, envelope, or bounds of parameters.For example, the network could have n number of outputs discretized overan associated range of some parameter or parameters. The network may betrained to indicate in some manner if the desired weld output isfeasible or not. In some embodiments, the network may also outputalternative similar welds that are feasible, for example instead offillet weld of some radius it could output a larger weld in order tocover some larger gap.

The neural network can also be trained to also output an indication ofthe resultant weld quality, weld shape, weld penetration, measure ofporosity, or a combination thereof. In some embodiments, the neuralnetwork can be any suitable neural network such as Dynamic Graph CNN,point-net, etc. The input data can be presented to the neural network inany suitable form (e.g., unordered list of vectors). The input data canbe pre-processed (e.g., using normalization).

In some embodiments, the input data can be provided to the neuralnetwork at once. Alternatively, the input data can be provided to theneural network incrementally. The neural networks can be trained in anysuitable manner. In some variations, the neural networks can includeLSTM modules or other similar modules meant to deal with time-varying orsequential data. The output could be in any suitable form. The neuralnetwork can be trained to output other information simultaneously, forexample, the probability of a successful weld.

In the second mode, the recognition and adaption can occur in one shot.In contrast to the first mode, there is no intermediary recognition orclassification step in the second mode. The inputs to the neural networkin this mode can include a model of the part(s) 112 or regions on thepart(s) 112. For example, the input to the neural network can be a CADmodel of a part(s) 112 that has had its surface sampled and representedas point cloud. The input to the neural network can further includeinformation about what the desired weld is and where it should be. Theoutput includes at least information about orientation and/or locationof the weld tip relative to the part.

In some embodiments, the neural network can also be trained such thatweld parameters, for example voltage, current, arc-length, wire/wirematerial, substrate material and thickness, and orientation of the seamrelative to gravity can be either inputs, outputs, or some combinationof both. For example, the user may want to constrain the voltage,current, and arc-length, (inputs the network) but not the orientation ofthe seam relative to gravity (output). Alterations or adaption of weldparameters like current, voltage, or arc-length can be useful forchanging the shape of the welds or handling transition betweensubstrates like welds that have been previously deposited. The neuralnetwork may be trained to take in a range, envelope, or bounds ofparameters. For example, there could be two inputs for voltage thatbound the allowable range. The network may be trained to output over arange, envelope, or bounds of parameters. For example, the network couldhave n number of outputs discretized over an associated range of someparameter or parameters. The network may be trained to indicate in somemanner if the desired weld output is feasible or not; it may also outputalternative similar welds that are feasible, for example instead offillet weld of some radius it could output a larger weld in order tocover some larger gap. The network may be trained to also output anindication of the resultant weld quality, weld shape, weld penetration,measure of porosity, or a combination thereof. In some embodiments, theneural network can be any suitable neural network such as Dynamic GraphCNN, point-net, etc. The input data can be presented to the neuralnetwork in any suitable form (e.g., unordered list of vectors). Theinput data can be pre-processed (e.g., using normalization).

In some embodiments, the input data can be provided to the neuralnetwork at once. Alternatively, the input data can be provided to theneural network incrementally. The neural networks can be trained in anysuitable manner. In some variations, the neural networks can includeLSTM modules or other similar modules meant to deal with time-varying orsequential data. The output could be in any suitable form. The neuralnetwork can be trained to output other information simultaneously, forexample, the probability of a successful weld.

In some embodiments, Simultaneous Localization and Mapping can be usedto fill in holes, gaps, or surface regions in the parts or seams (e.g.,if these were not initially scanned completely). In some embodiments,Simultaneous Localization and Mapping can be used to continue tolocalize in the case that CAD models are incomplete, and/or if a scan ofa part or seam misses a portion of the part and/or includes ambiguous orunreliable data.

In some embodiments, the controller 108 can continuously update models(e.g., kinematic models and/or CAD models) for example, by adding pointcloud data to these models. In some embodiments, the point cloud datacan include information associated with the addition of a weld that hasbeen deposited or created. Such data can, for example, be collected viathe laser emitter(s) illuminating a portion of the part behind the weldtip of the robot 110 as it travels. “Behind” here can refer to thedirection substantially opposite to the travel direction of the weld tipand/or a portion of the part recently passed by the weld tip.

The controller 108 can also add weld state data to models, for example,adding information about weld state based on point(s) or region(s) ofthe surface as observed using three-dimensional point cloud data. Inthis way, weld state data can be assigned such that information includesnot only the three-dimensional geometry (from the three-dimensionalpoint cloud), but also to some extent information about the weld (e.g.,based on CNN/ANN derived weld models). In some embodiments, the CADmodel can be continuously updated and can include notions of time orsome notion of sequence. Additionally or alternatively, CAD model can beupdated based on scanned data of the part(s) 112 during, between, orafter the welding process.

Although system 100 can operate with little or no user interaction, insome embodiments, system 100 can optionally include a user interface 106that can allow users to provide weld parameters and/or update weldparameters. The user interface 106 can enable a user to interact withthe system 100. Some non-limiting examples of the user interface 106include menu-driven interface, graphical user interface (GUI),touchscreen GUI, a combination thereof, and/or the like.

FIG. 4 is a flow diagram of a method 400 for precise welding with realtime feedback and dynamic adjustment in real time, according to anembodiment. At 402, the method includes receiving data of a workspace(e.g., workspace 101 in FIG. 1 ) including a part (e.g., part(s) 112 inFIG. 1 ) to be welded. The data can be sensor data from sensors (e.g.,sensor(s) 102 in FIG. 1 ) such as laser emitter(s), optical sensor(s),and/or a combination thereof. In some embodiments, these sensor(s) canbe synchronized before initializing the method 400. In some embodiments,data from different sensor(s) can be fused together and then analyzed.

At 404, the method includes transforming the data from the sensor intopoint cloud data. For example, the data from the sensors can betwo-dimensional data. This data can be transformed into athree-dimensional point cloud data representing the three-dimensionalsurface of the part.

At 406, the method includes determining desired state based on the pointcloud data. The desired state can be indicative of a desired position ofat least a portion of the welding robot (e.g., welding robot 110 in FIG.1 ) relative to the part. For instance, the desired state can beindicative of the desired portion of a weld tip included in the weldingrobot relative to the part. In some embodiments, an artificial neuralnetwork can be implemented on the point cloud data to determine thedesired state. In some embodiments, the desired state can be determinedby recognizing a seam to be welded on the part. In some embodiments, thedesired state can be determined by localizing the seam relative to thepart.

In some embodiments, the desired state can be determined by implementingan online classifier on the point cloud data. However, it may bechallenging to determine the desired state of portions of a part thatmay be occluded from the view of a laser emitter(s) using the onlineclassifier. In some embodiments, in response to determining suchocclusions, the desired state can be determined by implementing aparticle filter. In some embodiments, the desired state can bedetermined by implementing a combination of online classifier and aparticle filter.

At 408, the method includes comparing an estimated state to the desiredstate. The estimated state may be indicative of an estimated position ofat least a portion of the welding robot (e.g., weld tip) relative to thepart. The estimated state may be determined based on models such askinematic models and/or CAD models. A geometric comparator technique maybe implemented to compare the estimated state to the desired state.

At 410, the method includes updating welding instructions based on thecomparison. The welding instructions can be included in a robotic weldprogram and updating the welding instructions can include updating therobotic weld program. In some embodiments, updating the weldinginstructions can include updating a motion of the robot (e.g., movingthe robot from a previous position to a new position to reach thedesired state). In some embodiments, updating the welding instructionscan include updating a motion of motorized fixtures (e.g., fixture(s)116 in FIG. 1 ). In some embodiments, updating the welding instructioncan including updating weld parameters in real time. The weldinginstructions can also be updated based on a detection of a tack weldand/or detection of a variable gap on the seam to be welded.

It should be readily understood that method 400 can be continuouslyrepeated in any suitable manner. For example, prior to weldingoperations, welding instructions included in the robotic weld programcan include a motion/trajectory (e.g., weld path) for the welding robotand/or the motorized fixtures based on an initial estimated state or aninitial desired state. During the welding process, the sensors canprovide data of the workspace (e.g., part being welded, seam beingwelded, weld tip, etc.). Based on this data, an updated desired statecan be determined. This process can be continued until the weldingprocess comes to an end. Put differently, the weld path can becontinuously monitored and updated through the welding process.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. For example, some embodiments described herein reference asingle light source, a single detector, and/or a single pattern. Itshould be understood, however, that multiple light sources canilluminate a part. Each light source can illuminate the part with asimilar pattern, or different patterns. Patterns can be detected bymultiple detectors. In embodiments with multiple light sources, it maybe advantageous for each light source to illuminate the part with adifferent pattern such that detector(s) and/or compute devicesprocessing images captured by detectors can identify which light sourceprojected which pattern.

Furthermore, although various embodiments have been described as havingparticular features and/or combinations of components, other embodimentsare possible having a combination of any features and/or components fromany of embodiments where appropriate as well as additional featuresand/or components. For example, although not described in detail above,in some embodiments, methods of determining a shape of a portion of apart may include a calibration phase during which distortion of thedetector(s), the lens(es) on said detector(s), the distortion in thecombination of detector(s) and lens(es), and/or the relative position ofthe camera(s) to a test surface or fixture onto which a pattern(s) isprojected are determined.

Some embodiments described herein relate to methods and/or processingevents. It should be understood that such methods and/or processingevents can be computer-implemented. That is, where method or otherevents are described herein, it should be understood that they may beperformed by a compute device having a processor and a memory. Methodsdescribed herein can be performed locally, for example, at a computedevice physically co-located with a robot or local computer/controllerassociated with the robot.

Memory of a compute device is also referred to as a non-transitorycomputer-readable medium, which can include instructions or computercode for performing various computer-implemented operations. Thecomputer-readable medium (or processor-readable medium) isnon-transitory in the sense that it does not include transitorypropagating signals per se (e.g., a propagating electromagnetic wavecarrying information on a transmission medium such as space or a cable).The media and computer code (also can be referred to as code) may bethose designed and constructed for the specific purpose or purposes.Examples of non-transitory computer-readable media include, but are notlimited to: magnetic storage media such as hard disks, floppy disks, andmagnetic tape; optical storage media such as Compact Disc/Digital VideoDiscs (CD/DVDs), Compact Disc-Read Only Memories (CD-ROMs), andholographic devices; magneto-optical storage media such as opticaldisks; carrier wave signal processing modules, Read-Only Memory (ROM),Random-Access Memory (RAM) and/or the like. One or more processors canbe communicatively coupled to the memory and operable to execute thecode stored on the non-transitory processor-readable medium. Examples ofprocessors include general purpose processors (e.g., CPUs), GraphicalProcessing Units, Field Programmable Gate Arrays (FPGAs), ApplicationSpecific Integrated Circuits (ASICs), Digital Signal Processor (DSPs),Programmable Logic Devices (PLDs), and the like. Examples of computercode include, but are not limited to, micro-code or micro-instructions,machine instructions, such as produced by a compiler, code used toproduce a web service, and files containing higher-level instructionsthat are executed by a computer using an interpreter. For example,embodiments may be implemented using imperative programming languages(e.g., C, Fortran, etc.), functional programming languages (Haskell,Erlang, etc.), logical programming languages (e.g., Prolog),object-oriented programming languages (e.g., Java, C++, etc.) or othersuitable programming languages and/or development tools. Additionalexamples of computer code include, but are not limited to, controlsignals, encrypted code, and compressed code.

Where methods described above indicate certain events occurring incertain order, the ordering of certain events may be modified.Additionally, certain of the events may be performed concurrently in aparallel process when possible, as well as performed sequentially asdescribed above. Although various embodiments have been described ashaving particular features and/or combinations of components, otherembodiments are possible having a combination of any features and/orcomponents from any of the embodiments where appropriate.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the invention.However, it will be apparent to one skilled in the art that specificdetails are not required in order to practice the invention. Thus, theforegoing descriptions of specific embodiments of the invention arepresented for purposes of illustration and description. They are notintended to be exhaustive or to limit the invention to the precise formsdisclosed; obviously, many modifications and embodiments are possible inview of the above teachings. The embodiments were chosen and describedin order to explain the principles of the invention and its practicalapplications, they thereby enable others skilled in the art to utilizethe invention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that thefollowing claims and their equivalents define the scope of theinvention.

What is claimed is:
 1. A computer-implemented method for updatingwelding instructions for a welding robot, the computer-implementedmethod comprising: receiving, via at least one sensor, first data of apart to be welded; transforming, via a processor, the first data into athree-dimensional (3D) representation of the part; identifying, via theprocessor and based on the 3D representation, a desired state indicativeof a desired position of at least a portion of the welding robot withrespect to the part; comparing, via the processor, an estimated stateindicative of an estimated position of at least the portion of thewelding robot with respect to the part to the desired state, theestimated state based on a model of the unwelded part generated usingsecond data of the part, wherein the model includes annotationsrepresenting the welding instructions for the welding robot; andupdating, via the processor, the welding instructions based on thecomparison.
 2. The computer-implemented method of claim 1, whereinupdating the welding instructions include updating voltage parametersassociated with welding.
 3. The computer-implemented method of claim 1,wherein updating the welding instructions include updating powerparameters associated with welding.
 4. The computer-implemented methodof claim 1, wherein updating the welding instructions include updatingtrajectory of the welding robot based on the comparison.
 5. Thecomputer-implemented method of claim 1, wherein the second data includesimage data.
 6. The computer-implemented method of claim 1, wherein thesecond data is received via the at least one sensor.
 7. Thecomputer-implemented method of claim 1, wherein the second data isreceived via a second sensor.
 8. The computer-implemented method ofclaim 1, wherein the desired state is identified using an artificialneural network.
 9. The computer-implemented method of claim 1, furthercomprising updating the model based on the 3D representation.
 10. Thecomputer-implemented method of claim 1, wherein comparing the estimatedstate to the desired state comprises implementing a geometric comparatortechnique.
 11. The computer-implemented method of claim 1, furthercomprising updating the 3D representation based at least in part on thedesired state.
 12. The computer-implemented method of claim 1, whereinidentifying the desired state comprises inputting the 3D representationinto a neural network to generate the desired state.
 13. Thecomputer-implemented method of claim 1, wherein updating the weldinginstructions comprises adjusting a motorized fixture associated with thepart.
 14. The computer-implemented method of claim 1, wherein updatingthe welding instructions comprises dynamically updating weldingparameters in real time.
 15. The computer-implemented method of claim 1,further comprising: detecting a tack weld in a candidate seam on thepart to be welded; and updating welding parameters based at least inpart on the detection of the tack weld.
 16. The computer-implementedmethod of claim 1, further comprising: detecting that a gap associatedwith a seam on the part varies; and updating welding parameters based atleast in part on detection of the variable gap.
 17. Thecomputer-implemented method of claim 1, wherein updating weldinginstructions comprises updating welding parameters, and wherein theupdated welding parameters include at least one of welder voltage,welder current, duration of an electrical pulse, shape of an electricalpulse, and material feed rate.
 18. The computer-implemented method ofclaim 1, wherein receiving the first data of the part comprisesreceiving the first data in real time.
 19. The computer-implementedmethod of claim 18, further comprising monitoring the desired statebased at least in part on the first data received in real time.
 20. Thecomputer-implemented method of claim 1, wherein identifying the desiredstate further comprises implementing a particle filter on the 3Drepresentation.